Semver Impact of This PR

🟢 Patch (bug fixes)

_{🤖 This preview updates automatically when you update the PR.}

Codecov Results 📊

✅ 13 passed | Total: 13 | Pass Rate: 100% | Execution Time: 8.36s

All tests are passing successfully.

❌ Patch coverage is 20.00%. Project has 14846 uncovered lines.

Generated by Codecov Action

Is there any test coverage that we can add here to confirm this behaviour? Otherwise LGTM

In my view a regression test has limited use in this case because an integration should not be touching spans created outside the integration (unless it's for an agreed-upon reason, i.e., not duplicating spans between openai-agents and openai).

For the future, here's a script that verifies that demonstrates the edge case I'm addressing below. The messages argument is missing from the call to create() and the outer transaction is closed prematurely since it's status is INTERNAL_ERROR.

import sentry_sdk
from sentry_sdk.tracing import SPANSTATUS
from sentry_sdk.integrations.anthropic import AnthropicIntegration
from anthropic import Anthropic

sentry_sdk.init(
    dsn="",
    integrations=[AnthropicIntegration()],
    traces_sample_rate=1.0,
)

def main():
    client = Anthropic(api_key="dummy")
    with sentry_sdk.start_transaction(name="parent", op="test") as transaction:
        transaction.set_status(SPANSTATUS.INTERNAL_ERROR)
        try:
            client.messages.create(max_tokens=1024, model="model", stream=False)
        except Exception as e:
            print(f"Expected exception: {e}")

if __name__ == "__main__":
    main()

In my view a regression test has limited use in this case because an integration should not be touching spans created outside the integration (unless it's for an agreed-upon reason, i.e., not duplicating spans between openai-agents and openai).

I agree with what you're saying regarding an integration not touching spans created outside of it, but I think we could ensure that this edge case is resolved with a test that confirms that an anthropic span is present when the conditions that caused it are set.

To give a picture of what I'm envisioning (I used the script you've provided as the basis for it):

    sentry_init(integrations=[AnthropicIntegration()], traces_sample_rate=1.0)
    events = capture_events()

    client = Anthropic(api_key="z")
    client.messages._post = mock.Mock(return_value=EXAMPLE_MESSAGE)

    with start_transaction(name="anthropic") as transaction:
        transaction.set_status(SPANSTATUS.INTERNAL_ERROR)
        client.messages.create(
           # No messages value passed in
            model="model",
            max_tokens=1024,
            stream=False
        )

    (tx,) = events

    assert len(tx["spans"]) == 1
    span = tx["spans"][0]

    assert span["data"][SPANDATA.GEN_AI_SYSTEM] == "anthropic"

What do you think? Also happy to pair on this tomorrow when you're back online 😄